Skip to content

feat(bedrock): add TTL support to auto-injected tool and system/user cache points#2232

Open
kpx-dev wants to merge 1 commit intostrands-agents:mainfrom
kpx-dev:feat-ttl-tool-system-user-cache-points
Open

feat(bedrock): add TTL support to auto-injected tool and system/user cache points#2232
kpx-dev wants to merge 1 commit intostrands-agents:mainfrom
kpx-dev:feat-ttl-tool-system-user-cache-points

Conversation

@kpx-dev
Copy link
Copy Markdown
Contributor

@kpx-dev kpx-dev commented Apr 30, 2026

Description

Extends prompt caching TTL coverage beyond the user-supplied cachePoint path (already fixed in #1660) to the two SDK-managed auto-injected paths on BedrockModel.

Previously, only cache points that users passed directly inside message content could carry a ttl field — the formatter preserved it end-to-end. However, when the SDK auto-injected cache points on the user's behalf, the ttl was hardcoded away:

  1. cache_toolsBedrockModel(cache_tools="default") appends {"cachePoint": {"type": "default"}} to toolConfig.tools with no way to set a TTL.
  2. _inject_cache_point — when cache_config=CacheConfig(strategy="auto") is set, the SDK appends {"cachePoint": {"type": "default"}} to the last user message with no way to set a TTL.

Because Bedrock processes cache checkpoints in order toolConfig → system → messages and requires TTLs to be non-increasing, a user who wanted to run 1h caching end-to-end could not — the SDK-managed checkpoints would implicitly be 5m and violate the ordering rule (see #2121 Bug 2).

Changes

  • BedrockModel: added cache_tools_ttl: str | None config option. When set together with cache_tools, the TTL is propagated into the toolConfig cache point.
  • CacheConfig: added ttl: str | None field. When set, _inject_cache_point includes the TTL in the cache point appended to the last user message.

Users can now align all three cache checkpoint TTLs consistently:

from strands.models.bedrock import BedrockModel
from strands.models.model import CacheConfig

model = BedrockModel(
    model_id="us.anthropic.claude-haiku-4-5-20251001-v1:0",
    cache_tools="default",
    cache_tools_ttl="1h",
    cache_config=CacheConfig(strategy="auto", ttl="1h"),
)

API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CachePointBlock.html
Launch announcement: https://aws.amazon.com/about-aws/whats-new/2026/01/amazon-bedrock-one-hour-duration-prompt-caching/

Related Issues

Partially addresses #2121 (Bug 2: cache_tools ordering violation with 1h TTL).

Bug 1 from #2121 (user-supplied ttl being dropped) was resolved by #1660.

Documentation PR

Type of Change

New feature

Testing

Added 4 unit tests:

  • test_inject_cache_point_with_ttl — verifies CacheConfig(ttl="5m") propagates into the auto-injected message cache point.
  • test_inject_cache_point_without_ttl — backward compat: CacheConfig(strategy="auto") with no TTL still works.
  • test_format_request_cache_tools_with_ttl — verifies cache_tools_ttl propagates into toolConfig cache point.
  • test_format_request_cache_tools_without_ttl — backward compat: cache_tools without TTL still works.

Verified:

  • I ran hatch run prepare (lint, format, mypy all clean; pre-existing local env test failures unrelated to this change)
  • All 4 new tests pass

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@kpx-dev
Copy link
Copy Markdown
Contributor Author

kpx-dev commented Apr 30, 2026

Added 3 integration tests covering the new paths. Ran locally against us-east-1 with us.anthropic.claude-haiku-4-5-20251001-v1:0:

tests_integ/models/test_model_bedrock.py::test_prompt_caching_cache_tools_ttl               PASSED
tests_integ/models/test_model_bedrock.py::test_prompt_caching_cache_config_auto_with_ttl    PASSED
tests_integ/models/test_model_bedrock.py::test_prompt_caching_aligned_1h_ttl_across_checkpoints  PASSED

Coverage matrix:

Test What it proves
test_prompt_caching_cache_tools_ttl cache_tools_ttl is accepted by Bedrock on the toolConfig cache checkpoint without ValidationException.
test_prompt_caching_cache_config_auto_with_ttl CacheConfig(strategy="auto", ttl="5m") propagates TTL to the auto-injected message cache point and produces a cacheWrite.
test_prompt_caching_aligned_1h_ttl_across_checkpoints Regression test for #2121: 1h TTL on all three cache checkpoints (toolConfig → system → messages) satisfies Bedrock's non-increasing TTL ordering rule. This was the specific scenario that was impossible before this PR.

…cache points

Extends prompt caching TTL coverage beyond user-supplied cachePoint blocks
(PR strands-agents#1660) to the two SDK-managed auto-injected paths on BedrockModel:

- Adds cache_tools_ttl config option so the toolConfig auto-injected cache
  point can carry a TTL (e.g. '5m' or '1h').
- Adds ttl field to CacheConfig dataclass so _inject_cache_point propagates
  TTL into the cache point appended to the last user message when
  strategy='auto'.

Together, these let users align all three cache checkpoint TTLs (toolConfig
-> system -> messages) to satisfy Bedrock's non-increasing TTL ordering
rule -- which was previously impossible because cache_tools hardcoded an
implicit 5m TTL.

Partially addresses strands-agents#2121 (Bug 2: cache_tools ordering violation with 1h
TTL). Bug 1 from strands-agents#2121 was resolved by strands-agents#1660.

Tests:
- 4 unit tests covering cache_tools_ttl and CacheConfig.ttl with and
  without TTL (backward-compat).
- 3 integration tests against Claude Haiku 4.5 (officially documented for
  1h TTL on Bedrock), including a regression test that sets 1h TTL on all
  three cache checkpoints simultaneously.
- Model ID extracted into a _CACHE_TTL_MODEL_ID module constant so future
  model bumps are a one-line change.
@kpx-dev kpx-dev force-pushed the feat-ttl-tool-system-user-cache-points branch from 6a74d89 to 797f359 Compare April 30, 2026 23:33
@github-actions github-actions Bot added size/m and removed size/m labels Apr 30, 2026
@kpx-dev kpx-dev marked this pull request as ready for review April 30, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant